An Incremental Neural Network Construction Algorithm for Training Multilayer Perceptrons

The problem of determining the architecture of a multilayer perceptron together with the disadvantages of the standard backpropagation algorithm, directed the research towards algorithms that determine not only the weights but also the structure of the network necessary for learning the data. We propose a Constructive Algorithm with Multiple Operators using Statistical Test (MOST) for determining the architecture. The networks that are constructed by MOST can have multiple hidden layers with multiple hidden units in each layer. The algorithm uses node removal, addition and layer addition and determines the number of nodes in layers by heuristics. It applies a statistical test to compare different architectures. The results are promising and near optimal.